The contract with private windows is that they cleanup your navigation history and cookies. But they need to use your browser settings, otherwise you wouldn't be able to use the same keyboard shortcuts or the tabs would behave differently, for example.
You can create a new profile for what you want - those have a completely separate set of settings. Just click on the picture at the top right of the browser and you can easily alternate between them.